Questionable reliability of the speech-evoked auditory brainstem response (sABR) in typically-developing children.

Significant reliability correlation coefficients are bolded (p < 0.05). These data were taken from Hornickel et al. (2011a; Table 1). Hornickel et al., (2011a) have recently examined the test–retest consistency of speech-evoked auditory brainstem responses (sABRs) in “typically-developing children” both in quiet and in the presence of an acoustic background noise (4-speaker speech babble, 10 dB signal-to-noise ratio; using a six formant speech syllable/da/ as the stimulus). Based on the correlations between the tests taken at two points-in-time, they conclude that “reliability estimates were generally good.” They further conclude that the sABR may be an unique tool for the assessment of auditory-based communication skills in children. Clearly, the reliability of sABRs is an important issue based on the suggestions that: 1) this test be included as part of a battery for the diagnosis of central auditory processing disorders in children (Billet and Bellis, 2011), 2) it has a relationship to reading ability and music aptitude in children (Hornickel et al., 2012; Strait et al., 2011) and, 3) it may have predictive value in assessing reading ability and speech-in-noise perception in school-age children (Hornickel et al., 2011b). Although there are differences in the recommendations of experts (see Charter, 2003), a common view is that for use in clinical assessment, test–retest correlations above 0.80 are considered good and those below 0.70 are considered unacceptable (Cicchetti, 1994). Examination of the test-retest data from Hornickel et al. (2011a; Table 1), reconstructed and shown below (Table 1), reveals that only 1 of their 37 results is above 0.80 and only 2 are above 0.70. Thus, 34/37 (92%) of these correlations fail to reach a level commonly considered to be acceptable by clinical standards. Hornickel et al. (2011a) report that 21 of 37 correlations are significant. However for clinical assessment, it is the magnitude of the correlation which is the critical. It is not sufficient to show that a significant amount of the variance in test scores is common to individuals in both testing sessions. It is also necessary to show that a large percentage of this variance is consistent. However, the Spearman rank ordered correlation coefficient is not based on variance, so this measure is an ambiguous index of reliability. The importance of the actual magnitude of test–retest correlations was illustrated in a simulation study conducted by McFarland and Cacace (2006). This analysis showed that on the basis of reliability alone, at 0.80 the probability of correct diagnosis is about as likely as misdiagnosis (false alarms plus misses). At 0.70, either type of error is as likely as a correct diagnosis. Thus, unless a considerable portion of the variance in test scores is repeatable, diagnostic error is very likely. It is also important to stress that in addition to reliability, diagnostic accuracy also depends on the issue of validity (Cacace and McFarland, 1998; McFarland and Cacace, in press). Problems in diagnostic accuracy due to poor reliability are compounded when there are multiple opportunities for examinees to